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A visual scene is transformed free a very simple and convenient format, 

to an internal format which describes the same scene, but is more akin 

to complex manipulations. This format ia compatible with programs like 

"SEE". 

The entire analysis is done using a basic primitive which gives the 

orientation of a point with respect to a directed line. 

A novel handling of inaccuracies in the scene is achieved by considering 

the lines to be strips of small bur- not negligible width. The criterion 

is very general and easy to modify. 
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Tlie output of a program which processus the data from a vidiscctor 
is often a graph (list of vertices, their location and list of neighbors). 
This graph has to be checked for consistency such as all intersection 
of links are labelled as vertices, etc.. Further the regions have 
to be labeled and property lists which organize the information in 
very specialized forms for programs like "SEE" have to be prepared. 
A preprocessor called "SETUP" has been written in basic LISP. SETUP 
includes the following features. 

(a) Entire analysis is performed using a basic primitive which 
gives the orientation of a point with respect to a directed line, 
orientation is expressed U to tho left, or to the right or In the 
direction of the line or away from the directed line. Since in- 
accuracies in the Location of a vertex is unavoidable the directed 
line is in fact interpreted as a strip of variable width. Criterion 
used is very general but the organization is such that only one 
function has to be changed if a new criterion is to be used, 

(b) Since advice from lower level programs deal with only 
points (for example: points X and Y are on the same body) appropriate 
region names can be determined using available functions. 



(c) Main background is determined without additional information. 

It is interesting that the following can be achieved using only 
the "orient" primitive, 

(i) Determine whether a Riven point in inside or outside a 
general region (with holes). 

(11) Determine whether a given boundary encloses a region with 
bounded or unbounded area, 

(iii)Deteraine whether a given line segment intersects the boundary 
o£ a region. 

(iv) Determine the connectivity of a graph and relative location 
of the various subgraph*. 



1- Introduction 

Programs which recognize or understand scenes work with line 
diagrams of a scene, it is preferable for the data of the scene to 
be presented in a pseudo canonical form. This requires ordering of 
the neighbors of a vertex (for example, anticlockwise) and including 
the names of regions in describing the neighborhood of a vertex. 
Since one has to test a program (in turn a heuristic) for a large 



variety of scenes of considerable complexity preparation of data Ln 
Che pseudo canonical form can be very time consuming (see Figure 'HARD'). 
So it Is desirable to have a preprocessor which can accept the scene 
ln a simple format such as the incidence matrix. Further the output 
of programs that process the output of a vidiscctor is often ln the 
simplified format* thus the preprocessor can act as a link between 
the lower level programs and higher level programs. It is very 
desirable, why even necessary, for work on recognition programs to 
be able to use as data a line diagram (In black on white) of a scone 
to be analyzed. Thus one can test a program on a wide variety of 
scenes without too much trouble and facilitate answering the most 
often asked question of a recognition program: "What will the program 
do on thlfi scene? 11 . 

Instead of treating the preprocessor as merely a program (ol hack) 
it would be nice if recognition primitives and computational techniques 
Involved arc carefully selected ro that one can view the preprocessing 
as the computation of an automaton with certain capabilities. Such 
a view point can help in arriving at a hierarchy of automaton for 
understanding scenes. 



2. Obji-ct l vee 



2.1 General 

(a) The output of the program should be compatible with input 
for existing recognition program©* However better organization of 
output wherever desirable should also be considered. 

(b) Program ahould be in LISP and LISP features used should 

be general enough to run on almost any version of LISP, Compilation 
if desired should also be straight forward. 

(c) All functions defined should be prefixed by a special 
character to facilitate easy change of a name (H$ is used) in case 
of conflict that might arise when the program is made part of another 
program. 

(d) Remprop all unnecessary functions and variables at the end 
of the preprocessing. 

(c) The output will usually be in the form of property lists. 
Hence it should be possible to print out the output of the program in 
a format easy for human understanding. 

(f) All error processing should be centralized in a single 
routine (ttf ERROR) so that handling of an error can be altered very 



easily. This requirement is mandatory if the whole recognition system 
should somehow pracede in case of certain errors and not just hangup. 
(At present the preprocessor quits after printing a suitable message.) 



2.2 Particulars 

(a) The input of the graph of a scene should be as simple as 
possible. The data includes name of scene, and list of vertices, their 
location and unordered List of neighbors. 

(b) The output is in the form of property lists and 1 subset 
of the property lists is compatible with "SEE" of Guzman. 

(c) It is sometimes necessary to handle more than one scene 

with perhaps sane vertex names at the same time. This is made possible 
by including a feature by which names of vertices* regions and 
subgraphs can be prefixed by the name of the scene. All global 
properties are on the name of the rcene. 

/ 

(d) Check for certain types of constancies in thr Input such as; 

(i) Each link of the graph appears in two places, i.e. 
if B appears as a neighbor of A then A should appear as neighbor of 
B. The urge to relax this requirement (since finally a lower level 
program prepares input) and update suitably in case of error was curbed 
so that human errors in preparation of data is not misinterpreted. 



<ii) Intersection or meeting place of any two links should 
be at a vertex. 

(iii) Two or more neighbors of a vertex should not be 
collinear on the same side. (See also section 4.2). 

(c) Since input data and in particular locrttioo of a vertex is 
subject to hardware Inaccuracies, not to forget truncation errors, 
certain decisions such as the following should not be based on pure 
geometric theorems. (See also section 3.1.3). 

(1) Whether two links are in the same line. 

(ii) Whether three points are collinear. 

(£) Pind the connectivity of the Input graph. Separate the 
scene into subgraphs and find their relative location. This Is 
desirable since each subgraph can be a body or a hole. 

<g> Pind the main background (the only unbounded region which 
extends to infinity around the graph). The background of each sub- 
graph (the region in which the subgraph is situated) should also bo 
determined. The information on each subgraph should be organized in 
■uch a way that recursive analysis of the scene can be facilitated. 
Matching T*s cannot be found satisfactorily with tho available infor- 
mation, however, matching T's with respect to the main background 
should be determined. There should be facility for updating matching 
T's as and when more Is known about the scene. 



(h) It should be possible to change the name of a vertex, region 
or subgraph in a scene even after the preprocessing is complete, 
(Such as identifying vertex A on LEFT scene is same as vertex X in 
RIGHT scene of a stereo pair.) 

<i) Advice from lover level programs or between higher level 
programs such as 

(i) Point X {XCOR, YCOR) is in the background. 

(ti) Points X and Y arc on the sane body. 

(ili) Region which has KV* (ordered boundary) is . . . * 

<tv) Region with A and B (two ordered adjacent points on 

the boundary) on the boundary is 

should be updated by substituting internally generated names (for 
subgraphs or regions). This is facilitated by making available 
some utility functions which are retained even after unnecessary ones 
axe removed . 



3. InterestinR Features 

3.1 "Orient" Primitive 

In conforming with the intention to use staple recognition primi- 
tives it was felt that all the work involved in the preprocessor, re- 
ferred to as SETUP, can be done by using a primitive which answers the 
question: "What is the orientation of a point with respect to an 
oriented half line." The answer will be in the form: to the left, to 
the right, ahead or behind. This recognition ability is more basic 



than evaluating magnitude of angles. Further one always makes ap- 
proximations In Judging orientation and hence similar facility for 
approximating should be incorporated in this primitive. This ap- 
proximation Is all the more necessary in the computer program since 
location of vertices are subject to hardware errors. Truncation 
aggravates the problaa. 



3.1.1 Strict Mathematical Rule Such as: 

0* - ahead 

0* < 8 < 180* left 
180* behind 

180* < 6 < 360* right 
where 9 is the angle between the line and the line Joining the starting 
point of the oriented line and the given point (angle <AB) in Figure 1). 

This criterion is rejected because there is no room for approxi- 
mation and if this criterion is used no T's will be found in real scenes! 



3*1*2. This incorporates an approximation to criterion 3.1.1. 

-UP* 6 £ l<f ahead 
10* < 9 < 17(f left 
170* * O * 190* behind 
190* < 9 < 350 right 



this works fine except that very Large error Tcsults If the point Jf*t*«t 
from the origin of the directed line is large. In general one can 
tocpect that all vertices shift by about the sane amount Irrespective 
of their location duo to hardware inaccuracies, (barring edge effects). 
Hence aoving a vertex by large distance in order to consider it 
collinear with a line is not in general justified. 



3,1.3. The line is viewed as a strip of finite width for deciding 
collinearity. This in itself will not suffice because orientation 
of C with respect to Afl may not be colltnear whereas A is collinear 
with respect to BC (See Figure 1(b)). Even though successive 
logical extensions cannot be allowed in approximate techniques at 
least three points should be Judged collinear no Batter which two of 
the three points is used for the directed line. So three points arc 
iudged collinear if any one point is collinear with respect to the 
strip along the other two points, I.e. A t B and C are considered if: 
C lies in a strip of width xEPS along AB 
or B lies in a strip of width ±EPS along AC 
or A lies in a strip of width ±KPS along 6C 

It looks at the outset that taking the strip parallel to one of 
the three sides of the triangle is not general enough. In fact one 
would like to fit a line which minimizes the sum of the squares of 
the distance from each point to the Line. And the three points are 
collinear If they lie within a strip of width ±EPS about this line. 



It is not easy to find such a line. (This is not the regular least 
square fit lino). Further, it can be easily shown that, if three 
points arc col linear from the host fit line crltcrLon, it will also 
be collinear with respect to our three strip criterion. 

Figure 2(a) shove the criterion used in the present preprocessor 
(MtfSETUP)* The organisation of the preprocessor is such that one 
easily can use any other criterion by rewriting HXOXHT. Figure 2(b) 
illustrates the case where EPS is larger than distance between A and B. 
Obviously greater reliance is put in direction of AC or BC than that 
of AB. 

It is reasonable to use a value for the half width of the strip 
(EPS) which is commensurate with the hardware Inaccuracies. In- 
creasing EPS beyond reason (half of minimum distance between any two 
neighbors) will cause trouble which is understandable. Ail attempts 
to adjust value of EPS dynamically by studying only the neighborhood 
of a vertex should be considered carefully* One of the dangers 
of such an adjustment is that many vertices will end up being T'sl 

One interesting technique of analysis of a scene would be analyze 
it at more than one value of EPS and reconcile the discrepancy if any. 

It should be noted that MWRKT only shows conclusions about the 
orientation and does not adjust the coordinates of the vertices to 
fit the orientation geometrically (according to 3.1.1). It is 
feared that any attempt to adjust data recursively will ruin the 
features. 



3.2 HBP Function 



A primitive derived fronTorient finds whether two line segments 



Intersect or not (Figure 3), 



3,3 "Inside" Function 

It IB Interesting that we can determine whether a point is 
internal to a region (given by ordered vertices on the boundary) by 
using the orient primitive. A point Is inside if the number of 
intersections of the boundary with the infinite half line Joining 
the point to a vertex on Che boundary is add, otherwise outside. 



3,4 "Bounded" Function 

Often we need to find If a region given by an ordered set of 
vertices has bounded area. The technique first finds the vertex 
with the minimum X coordinate. Then if the region is to the left of 
this point it is unbounded (Figure 2). 



4* Overall Technique (MigETUPX) 

The names referred to in this memo (prefixed by H7H arc names 
af LIS? function!. 



4.1 M% SETUP 1 

Input data is separated after appending a prefix (NIL if M% SETUP if 
used and NAME if MV, SETUP* i« used). 



4.2 MftARRAHGE 

The neighbors of each vertex arc arranged in counter clockwise 
order. This is achieved by successively adding a neighbor to the 
ordered list. If EPS2 (square of the half width EPS for M^ORNT) Is 
large two vertices will fall in the same direction which is an error. 



4.3 M'/TYPE 



The TYPE* (see section 5) o£ each vertex is found. 



4.4 M'AFREGION 

Regions associated with each boundary is labeled. The technique 
is; start with a vertex A (Figure TEST) and choose a neighbor (say H) 
connected to a A by an unused link {i*e* AH has not yet been identified 
as a boundary of a region). Hark ffi as used. At II pick the vertex (KJ) 
which is before A in the ordered list of neighbors of H. HK1 should alsc 
be unused which will not bo the case if data is not consistent. So wc 



have AHKl as a partial boundary of a region :4. Proceed as before 
from Kl until we reach A. There make sure that the next link {after M&) 
is indeed AH, This will not be the case if we had started from H 
instead of A, So proceed until the first link used is obtained again. 
It should be noted that a vertex can appear more than once on its 
boundary (:4 has as K vertices H,KitHpJ,I,H,G,F»E,D > C>B»M > A ), At 
this stage the region names and their boundaries are not final for 
those regions which have holes. For example: entire area within 
A2 A7 A6 A3 (Figure CRAZY) will be :3 and entire area outside Dl DA 
D3 D2 will hove a separate number. Later the extra name is eliminated 
and the boundary of: 3 is adjusted. 



4.5 M % SUBGRAPH 

Connectivity of the total graph Is found. The technique is that 
each subgraph is constituted by a minimal subset of vertices whose 
neighbors are also in the subset. For example, 61, G2 and G3 
(Figure CRAZY) form a subgraph. 



4.6 M "/.SEPARATE 

The relative location (structure) of the various subgraphs arc 
determined* The technique is: the background (unbounded region 
associated with a subgraph) of a subgraph is determined (Section 3,4). 



Pick the subgraph which has the vertex with the minimum X coordinate 
of all the vertices {Al of SUBGRAPH) in Figure CRAZY). The background 
of the scene is the background of Subgraf 10 (:2). Separate che 
other subgraphs according as OUTSG if one point on than is in :2 
(i.e. outside SUBGRAG10 and INSG if one point in than is not in :2. 
For example subgraphs 7, 4 and 6 will be in OUTSG and 3.2.5 and 1 in 
INSC^i Separate INSG into subsets within each region of SUBGRAF10. 
Proceed with OUTSG and various subsets of INSG from beginning (re- 
cursively) except that the background in each case is to be merged 
with a region of SUBGRAF10. For example background of OUTSG {aasociatod 
with CI C2 C3) will have to be merged with :2 and background o£ 1.2 and 
5 (associated with Fl F2 F3 FA) is to be merged with :1. The structure 
information is organized as a list and is explained in Figure CRAZY. 

A note about finding the background (M%BGR0UND). Any cardinal 
point (i.e. a vertex with minimum x or y coordinates or with maximum 
X or Y coordinates )can be used for finding the background. The region 
name in each case should be the same. However, if the intersection 
of two links is not identified as a vertex the region names will be 
different causing an error exit. 



4.7 MXADJUST1 

Merging information of multiple region names are gathered and 
information about the subgraphs in each region is collected. 



4.8 MftRHUND 

If there is more than one name possible fox a region one with a 
lower number is retained. The property list of region names are 
updated (take care of multiple boundaries) and property lists of 
unused region names arc deleted. However, the occurrences of 
region names eliminated have yet to be replaced by corresponding 
names. A list (M LIST) of substitutions necessary is prepared. 



4.9 M^REPLACE 

H LIST is used to make the substitutions in all property list: 
associated with M%SETDT (or M/SSETUP*). 



4.10 H7-SEEDAIA 

Since one main requirement of H%SETUP was to be compatible with 
"SEP* appropriate new properties are created. NAME is given a value 
which is compatible for using "PRRPARA" (a preprocessor for SEE). 
However, PREPARA can be bypassed. Value of HAKE can be used to plot 
the scene using a package written by Guzman. 



4.11 M>>MATCHT 



A pair of T's in a scene are matching T's if 



(i) they face eachother with their tails collinear and 
are the closest possible, 

(ii) the base region (one which subtends an angle of 180* 
at the vertex) of each T should not be a background, and 

(iii)the line joining them does not intersect any region 
identified as background. 

MXSETUP finds the matching T's only with respect to the main 
background (I.e. 35 in Figure HAKD). However, M%MATQIT Is so 
organized that further updating can be done very easily. For 
example, M%MAXCHT can be called using 36 and 34 as background and 
operating on only those T's which already have mates. In this case 
it well not make an alteration in the set of matching T's. MrJiATCMT 
was tested on HARD by identyfying 26 (rather illogical) as background. 
Then vertices a, fa and b, fb were eliminated from the set of matching 
T's. 

The technique used i€ to make sure that each T(of a matching pair) 
accepts the other as a mate, When all T's are available for matching. 
Since H%0RtfI uses a strip for matching coUinearliy sometimes, cycle 
can arise which is an error. 



5. Property Lists 

H%SKTDP (or M'/SETUP*) sots up a number of property lists whose 
description follows. 



5.1 NAME 

Its value 15 data suitable for rMPftM, 
PREFIX* 



ACCURACY 
ERROR 

VERTICES 

REGIONS 

CONNECTIVITY 

SUBGRAPHS 

HAXMIN 

BACKGROUND 

STRUCTURE 



Explode list of characters in HAKE if M '/.SETUP* 
is used, otherwise NIL. 

Half width of the atrip for MKORKT. 

Error message in case M^MAIQET had to delete vertices 
from matching because of cyclicity. (Such as B is 
mate of A» C is mate of B, and A is mate of C. 1 

List of vertices in the scene. 

List of regions in the scene. 

Number of subgraphs in the scene. 

List of subgraphs in the scene. 

List of cardinal points (Min X, Max X, Min Y, Max Y). 

List of backgrounds (only one Initially). 

Structural information as explained in Figure CRAZY. 



5.2 Nth Subgraph (SGRAFfO 



MAXMIN | 

VERTICES I 

REGIONS f 
BACKGROUND 



Seme as in case of Name (5.1). 



5.3 VEKTEX 

XCOR 
YCOR 
TYPE 

KIND 

N VERTICES 
N REGIONS 
HEXTB 



X-Coordinate 

Y- Coordinate 

Type information. See references to GUZMAN for 

details. 

List of counter-clockwise neighboring vertices and 

regions (alternately). 

List of neighboring ver tice s. 

List of regions having VERTEX on the boundary. 

Name of matching T, otherwise NIL. 



COOR 
TYPE* 

y:< 



CROSS 



List of XCOK and YCOR. 

Classification of the vertex. Types L> ARROW, PEAK, X, T 3 

HULTX, K» FORK are the same as GUZMAN new types. 

One straight line and more than two branches on one 

side (vertex G in Figure TEST)* 

Straight line (I in TEST). 

Two straight lines cross at the vertex (B in TEST)* 



KK becomes HULTI, CROSS becomes X in TYPE, however, SL continues to 



be SL which may cause error in SEE. However, 
not expect SL type of vertices. 



as it stands does 



OVERTICES List of elements each of which is a list of a neighbor 
and an orientation* Orientation gives a measure of the 
angle to be rotated (anticlockwise) from the neighbor 
to reach the next neighbor. For example at A in TEST 
MAB<180* is denoted L, H&P180* <R) and at G FGH-18tf <CA). 



5.4 REGION 



FOOP See GUZMAN 

NEIGHBORS List of adjoining regions. 

KVERTICES List of all vertices on the boundary. Ho separation 

even if there are multiple boundaries. 
BACKGROUND Has 'BACKGROUND'* if region is a background, otherwise NIL 

< :4 In TEST ). 

BOUNDARY List of el^nents. Each element is a list of links 

in one branch of the boundary. Each link is a list 
of ordered (counter-clockwise) neighbors on a seg- 
ment of the boundary (;1 and :2 In TEST). 

OTRAVERSE List of elements. Each element is a list of subclcmcnts 
in one branch of the boundary. Each subelemcnt is 
a list of a vertex on the boundary and a measure of 
the angle subtended by the region at that vertex. 

NEIGHBORS* List of elements. Each element is the ordered NEIGHBORS 
of one branch of the boundary. 

KVERTICES* List of elements. Each clement is the ordered KVERTICES 
of one branch of the boundary. 

INSUBGRAPH List of subgraphs located in the region. 



can be done once the two OTRAVERSE'S arc compatible. OVEKTICES con 
also be used similarly to vertices. 

(vlii) Choose accuracy very carefully. 

(ix) Since each subgraph can be a hole or a body»soparatc 

analysis o£ a subgraph is facilitated by the properties on each 
subgraph. 

(x) STRUCTURE can bo used to match scenes at a gross level. 
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6. Die o£ H^SETUT 



(i) Use HV-SETUP* if more than one eccne with same vertex names 
have to be handled simultaneously. 

(ii) Even If very little Information is changed in the scene it 
will have to be set up again. 

(iii)The property lists created by M^SETUP can be printed out 
by using MtfANSWER. 

(iv) The property Lists can be ste* red on dLek on disk in a 
suitable internal format so that the property lists can be reestablished 
by reading the file. Consult GUZMAN for the program. 

(v) Since M'/rSETUP finds matching T's only with respect to the 
main background updating is necessary every time background infor- 
mation is altered. This should be all right since one may not ana- 
lyze scenes with holes in the main background. Hence In general 
matching T's will be a subset of the matching T's found by M^.SETUP. 

(vi> Advice from the lover level programs can be updated by using 
the special functions available (See SETUP Comment). 

(vii)Matchlng models of regions la facilitated by the property 
OTRAVERSE, Exact comparison of lengths of sidos and magnitude of angles 
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